{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 神经网络的学习\n",
    "## 4.1 从数据中学习\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.2 损失函数\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 均方误差\n",
    "$$\n",
    "E= \\frac{1}{2}\\sum_{k}(y_k -t_k)^2\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.5975"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "def mean_squared_error(y,t):\n",
    "    return 0.5*np.sum((y-t)**2)\n",
    "\n",
    "t = [0,0,1,0,0,0,0,0,0,0]\n",
    "y = [0.1,0.05,0.6,0.0,0.05,0.1,0.0,0.1,0.0,0.0]\n",
    "\n",
    "mean_squared_error(np.array(y),np.array(t))\n",
    "\n",
    "y = [0.1,0.05,0.1,0.0,0.05,0.1,0.0,0.6,0.0,0.0]\n",
    "mean_squared_error(np.array(y),np.array(t))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.2.2 交叉熵误差\n",
    "交叉熵误差对错误的惩罚力度更大。\n",
    "$$\n",
    "E = -\\sum_{k}t_k\\log y_k\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2.302584092994546"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "def cross_entropy_error(y,t):\n",
    "    delta = 1e-7\n",
    "    return -np.sum(t*np.log(y+delta))\n",
    "t = [0,0,1,0,0,0,0,0,0,0]\n",
    "y = [0.1,0.05,0.6,0.0,0.05,0.1,0.0,0.1,0.0,0.0]\n",
    "cross_entropy_error(np.array(y),np.array(t))\n",
    "y = [0.1,0.05,0.1,0.0,0.05,0.1,0.0,0.6,0.0,0.0]\n",
    "cross_entropy_error(np.array(y),np.array(t))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.2.3 mini-batch学习\n",
    "对所有训练的数据进行学习，使得损失函数的值尽可能的小。但是这种方法在实际训练过程中，计算量非常大，所以采用mini-batch的方法，抽取部分训练数据求出误差函数的值。\n",
    "\n",
    "$$\n",
    "E = -\\frac{1}{N}\\sum_{n}\\sum_{k}t_{nk}\\log y_{nk}\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(60000, 784)\n",
      "(60000, 10)\n",
      "[33975 48445 31187 20079 13493 30516 40769  7915 27493 17357]\n",
      "(10, 784)\n",
      "(10, 10)\n"
     ]
    }
   ],
   "source": [
    "import sys, os\n",
    "sys.path.append(os.pardir)  # 为了导入父目录的文件而进行的设定\n",
    "import numpy as np\n",
    "from dataset.mnist import load_mnist\n",
    "\n",
    "(x_train, t_train),(t_text, t_test) = \\\n",
    "    load_mnist(normalize=True, one_hot_label=True)\n",
    "print(x_train.shape)\n",
    "print(t_train.shape)\n",
    "\n",
    "# 从这写训练数据中随机抽取10笔数据\n",
    "train_size = x_train.shape[0] \n",
    "batch_size = 10\n",
    "batch_mask = np.random.choice(train_size, batch_size)\n",
    "x_batch = x_train[batch_mask]\n",
    "t_test = t_train[batch_mask]\n",
    "\n",
    "print(batch_mask)\n",
    "print(x_batch.shape)\n",
    "print(t_test.shape)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.2.4 mini-batch版交叉熵误差的实现\n",
    "主要是用来批量处理数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ndim\n",
      "0.510825457099338\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "def cross_entropy_error(y,t):\n",
    "    if y.ndim == 1:\n",
    "        print(\"ndim\")\n",
    "        t = t.reshape(1, t.size)\n",
    "        y = y.reshape(1, y.size)\n",
    "    batch_size = y.shape[0]\n",
    "    return -np.sum(t *np.log(y +1e-7)) / batch_size\n",
    "y = np.array([0.1,0.05,0.6,0.0,0.05,0.1,0.0,0.1,0.0,0.0])\n",
    "t= np.array([0,0,1,0,0,0,0,0,0,0])   \n",
    "\n",
    "t_ss  = cross_entropy_error(y,t)\n",
    "print(t_ss)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.3 数值微分\n",
    "\n",
    "### 4.3.1 导数\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.3.2 数值微分的例子\n",
    "\n",
    "$$\n",
    "y = 0.01x^2 + 0.1x\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.1999999999990898\n",
      "0.2999999999986347\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAioAAAHHCAYAAACRAnNyAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAARxVJREFUeJzt3Xd4FHXix/HPplOS0FJJCKGFEmooUhQVFBEVBBEVlWZBUVRORTwL/DwFy6mn54FyUlXEBghYQZp0EnoJNQVIQk9CQtru/P5AckYCJCHJbHm/nifP487OTj7D7GY+zs58x2IYhiEAAAA75GZ2AAAAgEuhqAAAALtFUQEAAHaLogIAAOwWRQUAANgtigoAALBbFBUAAGC3KCoAAMBuUVQAAIDdoqgALmrjxo3q0qWLqlWrJovFoi1btpgd6SL169fX0KFDzY4BwEQUFcAF5efna+DAgTp16pTee+89zZ49WxEREaZkWbNmjcaPH68zZ86Y8vtL4vXXX9cdd9yhoKAgWSwWjR8/3uxIgMvwMDsAgMp34MABJSYmaurUqXrooYdMzbJmzRpNmDBBQ4cOVY0aNYo8Fx8fLzc38/9/6qWXXlJwcLDatm2rn3/+2ew4gEuhqAAu6NixY5J0UTGwN97e3mZHkCQdOnRI9evX14kTJxQQEGB2HMClmP+/KgAq1dChQ9W9e3dJ0sCBA2WxWHT99dcX/hQ3f/369QsfJyQkyGKx6J133tEnn3yihg0bytvbWx06dNDGjRsvev2ePXt09913KyAgQFWqVFFUVJT+/ve/S5LGjx+v5557TpIUGRkpi8Uii8WihIQEScWfo3Lw4EENHDhQtWrVUtWqVXXNNddo8eLFReZZvny5LBaLvvrqK73++usKCwuTj4+PevToof3795f63+zP6w+gcnFEBXAxjz76qOrWras33nhDo0ePVocOHRQUFKTXX3+9VMv54osvlJmZqUcffVQWi0VvvfWW+vfvr4MHD8rT01OStG3bNl177bXy9PTUI488ovr16+vAgQNauHChXn/9dfXv31979+7VnDlz9N5776lOnTqSdMmjFmlpaerSpYuys7M1evRo1a5dWzNnztQdd9yhb775RnfeeWeR+SdNmiQ3Nzc9++yzSk9P11tvvaXBgwdr/fr1ZfiXA2AGigrgYjp37qzc3Fy98cYbuvbaa3XXXXdJUqmLSlJSkvbt26eaNWtKkqKiotS3b1/9/PPPuu222yRJTz75pAzDUFxcnOrVq1f42kmTJkmSWrVqpXbt2mnOnDnq16/fFY9cTJo0SWlpaVq1apW6desmSXr44YfVqlUrjRkzRn379i1yTktOTo62bNkiLy8vSVLNmjX11FNPaceOHYqOji7V+gIwB1/9ACiTQYMGFZYUSbr22mslnf9qRpKOHz+ulStXavjw4UVKiiRZLJYy/c4ffvhBHTt2LCwpklS9enU98sgjSkhI0K5du4rMP2zYsMKSUlxGAPaPogKgTP5aPi6UltOnT0v6XxkozyMXiYmJioqKumh6s2bNCp8vTUYA9o+iAkDSpY9yWK3WYqe7u7sXO90wjHLLdLUcISOAy6OoAJB0/mhDcYOu/fUoRUk1aNBAkrRjx47Lzlear4EiIiIUHx9/0fQ9e/YUPg/AuVBUAEiSGjZsqD179uj48eOF07Zu3arVq1eXaXkBAQG67rrrNG3aNCUlJRV57s9HNKpVqyZJJRqZ9tZbb9WGDRu0du3awmlZWVn65JNPVL9+fTVv3rxMWQHYL676ASBJGj58uN5991316tVLI0aM0LFjxzRlyhS1aNFCGRkZZVrmBx98oG7duqldu3Z65JFHFBkZqYSEBC1evLjw3kIxMTGSpL///e+655575Onpqdtvv72wwPzZCy+8oDlz5qh3794aPXq0atWqpZkzZ+rQoUP69ttvK2wU29mzZysxMVHZ2dmSpJUrV+of//iHJOmBBx7gSA5QgSgqACSdPyF11qxZeuWVVzRmzBg1b95cs2fP1hdffKHly5eXaZmtW7fWunXr9PLLL2vy5MnKyclRRESE7r777sJ5OnTooNdee01TpkzRTz/9JJvNpkOHDhVbVIKCgrRmzRqNHTtWH374oXJyctSqVSstXLhQffr0KeuqX9Gnn36qFStWFD5etmyZli1bJknq1q0bRQWoQBaDs8oAAICd4hwVAABgt/jqB4BLOnfunNLT0y87T61atYoMGAeg8lFUALikuXPnatiwYZedZ9myZcXeqBFA5eEcFQAuKSUlRTt37rzsPDExMUVuEwCg8lFUAACA3eJkWgAAYLcc+hwVm82mo0ePytfXt8x3YwUAAJXLMAxlZmYqNDT0igM1OnRROXr0qMLDw82OAQAAyiA5OVlhYWGXncehi4qvr6+k8yvq5+dnchoAAFASGRkZCg8PL9yPX45DF5ULX/f4+flRVAAAcDAlOW2Dk2kBAIDdoqgAAAC7RVEBAAB2i6ICAADsFkUFAADYLYoKAACwWxQVAABgt0wvKkeOHNH999+v2rVrq0qVKmrZsqU2bdpkdiwAAGAHTB3w7fTp0+ratatuuOEG/fjjjwoICNC+ffu4rToAAJBkclF58803FR4erunTpxdOi4yMNDERAACwJ6Z+9fP999+rffv2GjhwoAIDA9W2bVtNnTr1kvPn5uYqIyOjyA8AAHBephaVgwcPavLkyWrcuLF+/vlnPfbYYxo9erRmzpxZ7PwTJ06Uv79/4Q93TgYAwLlZDMMwzPrlXl5eat++vdasWVM4bfTo0dq4caPWrl170fy5ubnKzc0tfHzh7ovp6enclBAAgHK2dHeabogKlJvblW8eWBoZGRny9/cv0f7b1CMqISEhat68eZFpzZo1U1JSUrHze3t7F94pmTsmAwBQceZsSNKImZv06GexstlMO6ZhblHp2rWr4uPji0zbu3evIiIiTEoEAAA2JZzSKwt2SJJah/mX+xGV0jC1qDzzzDNat26d3njjDe3fv19ffPGFPvnkE40aNcrMWAAAuKyU9HMa+Vmc8q2Gbm0ZrFE3NDI1j6lFpUOHDpo3b57mzJmj6Ohovfbaa3r//fc1ePBgM2MBAOCScvKtGjk7VifO5qppsK/evqu1LBbzjqZIJp9Me7VKczIOAAC4NMMw9Levt+q7uCOqUdVTC5/opvBaVSvkdznMybQAAMA+TFudoO/ijsjdzaKP7mtXYSWltCgqAAC4uFX7juv1xbskSS/e2kxdG9UxOdH/UFQAAHBhB4+f1ajP42QzpAHtwjS8a32zIxVBUQEAwEVl5OTroVmblJFToHb1auiN/tGmnzz7VxQVAABckNVm6MkvNuvg8SyF+PtoygMx8vZwNzvWRSgqAAC4oDd/2qMVe4/Lx9NNUx9sr0BfH7MjFYuiAgCAi/km9rA+WXlQkvTOwNaKrutvcqJLo6gAAOBC4pJO68XvtkuSnryxkW5rFWpyosujqAAA4CJS0s/pkVmxyrPadHPzID3Ts4nZka6IogIAgAs4l2fVI7P+Nzz+e4PamHqzwZKiqAAA4OQMw9Dz327T9iPpqlXNS1MfbK9q3h5mxyoRigoAAE7uo2X7tXDrUXm4WfSfwfYzPH5JUFQAAHBiv+xM1Tu/7JUk/V/faF3ToLbJiUqHogIAgJPak5qhp+dukSQ92DlC93WqZ26gMqCoAADghE5l5emhmZuUnWdVl4a19fJtzc2OVCYUFQAAnExegU2PfRarw6fPKaJ2VX10Xzt5ujvmLt8xUwMAgEuasHCn1h86pereHpr6YHvVrOZldqQyo6gAAOBEZq1N0Ofrk2SxSP+6p42aBPmaHemqUFQAAHASK/Ye14SFuyRJz/WKUo9mQSYnunoUFQAAnMC+tEw98XmcrDZD/dvV1WPdG5odqVxQVAAAcHAnz+Zq+MyNyswtUIf6NTWxf0tZLPY/PH5JUFQAAHBguQVWjfwsVsmnzqlerar6+IH28vZwNztWuaGoAADgoAzD0Ljvtmtjwmn5+nho2tD2quXAV/gUh6ICAICD+s/yA/ou7ojc3Sz66L52ahTo2Ff4FIeiAgCAA/ppR4re/jlekjT+jha6rkmAyYkqBkUFAAAHs/1weuE9fIZ2qa8HrokwN1AFoqgAAOBAUtNz9NCsjcrJt6l7kwC91KeZ2ZEqFEUFAAAHkZ1XoBEzNyotI1dNgqrrw/vaysNB7+FTUs69dgAAOAmbzdAzc7do59EM1a7mpU+HdJCfj6fZsSocRQUAAAfw9i/x+nlnmrzc3fTJgzEKr1XV7EiVgqICAICd+3pTsiYvPyBJeuuuVoqJqGVyospDUQEAwI6tP3hSL87bLkl68sZG6te2rsmJKhdFBQAAO5V4MksjP4tVvtVQn5YheqZnE7MjVTqKCgAAdig9O1/DZ2zU6ex8tQ7z1zsDW8vNzTluNFgaFBUAAOxMXoFNIz+L1YHjWQrx99HUB9uripfz3GiwNCgqAADYkQs3Glx78KSqe3to2tAOCvTzMTuWaSgqAADYkQ9/269v4w6fv9Hg4HZqFuJndiRTUVQAALAT8zcf0bu/7pUkvdY3Wt2d9EaDpUFRAQDADqw/eFLPf7NNkvTodQ10X6d6JieyDxQVAABMduD4WT0yO1Z5VptubRmssbc0NTuS3aCoAABgopNnczVs+kaln8tX23o19O7dbVzyMuRLoagAAGCSnHyrHp61SUmnshVeq4qmPthePp6ueRnypVBUAAAwgc1m6G9fbVVc0hn5V/HU9KEdVae6t9mx7A5FBQAAE7z1c7wWb0+Rp7tFHz8Qo0aB1c2OZJcoKgAAVLI5G5I0ZcX/7oZ8TYPaJieyX6YWlfHjx8tisRT5adqUM50BAM5rxd7jemn+DknS0z0b6862YSYnsm8eZgdo0aKFlixZUvjYw8P0SAAAVIjdKRka9XmcrDZD/dvV1VM9Gpsdye6Z3go8PDwUHBxsdgwAACrU0TPnNGz6Rp3NLdA1DWppUv9Wsli4DPlKTD9HZd++fQoNDVWDBg00ePBgJSUlmR0JAIBylX4uX0Onb1BqRo4aBVbXx/e3l5eH6btgh2DqEZVOnTppxowZioqKUkpKiiZMmKBrr71WO3bskK+v70Xz5+bmKjc3t/BxRkZGZcYFAKDUcgusGjk7VnvTzirQ11szhnWQf1VPs2M5DIthGIbZIS44c+aMIiIi9O6772rEiBEXPT9+/HhNmDDhounp6eny83Ptu0sCAOyPzWZozFdbNH/LUVXzctdXIzurRai/2bFMl5GRIX9//xLtv+3quFONGjXUpEkT7d+/v9jnx40bp/T09MKf5OTkSk4IAEDJvf1LvOZvOSoPN4sm3x9DSSkDuyoqZ8+e1YEDBxQSElLs897e3vLz8yvyAwCAPZq9LlGTl58fK2Vi/5a6rkmAyYkck6lF5dlnn9WKFSuUkJCgNWvW6M4775S7u7vuvfdeM2MBAHBVft2VplcXnB8rZcxNTTSwfbjJiRyXqSfTHj58WPfee69OnjypgIAAdevWTevWrVNAAK0TAOCYNied1pNz4mQzpHs6hOvJGxuZHcmhmVpUvvzySzN/PQAA5SrhRJZGzNyknHybro8K0D/6RTNWylWyq3NUAABwVCfP5mrI9A06lZWn6Lp++ui+dvJwZzd7tfgXBADgKp3Ls2rEzE1KPJmtsJpVNG1oB1XzNn3wd6dAUQEA4CpYbYaenLNZW5LPqEZVT80c3lGBvj5mx3IaFBUAAMrIMAyN/36nluxOk5eHm/77YHs1DKhudiynQlEBAKCM/v3bfs1elyiLRXp/UBu1r1/L7EhOh6ICAEAZfLkhSf/8da8kafztLXRry+IHK8XVoagAAFBKv+5K04vztkuSRt3QUEO61Dc3kBOjqAAAUAqxiaf0xBfnB3QbGBOmZ2+OMjuSU6OoAABQQvvSMjV8xiblFth0Y9NATezfkgHdKhhFBQCAEkhJP6cHp21Q+rl8ta1XgwHdKgn/wgAAXEF6dr6GTNuglPQcNQyopmlDOqiKl7vZsVwCRQUAgMvIybfqoVkbtTftrIL8vDVzeEfVrOZldiyXQVEBAOASCqw2PTlnszYmnJavj4dmDu+osJpVzY7lUigqAAAUwzAMvbxgp37d9b9RZ5sG+5kdy+VQVAAAKMb7S/ZpzoYkuVmkD+5po04NapsdySVRVAAA+IvP1iXqX0v3SZL+r2+0bolm1FmzUFQAAPiTH7an6JUFOyRJo3s01v3XRJicyLVRVAAA+MPv+07o6S+3yGZI93YM1zM9G5sdyeVRVAAAkLQl+Ywemb1JeVabekcH6x/9GHXWHlBUAAAub/+xTA2dvkHZeVZ1a1RH79/TRu5ulBR7QFEBALi0w6ezdf9/N+hMdr5ah9fQxw/EyNuDUWftBUUFAOCyTp7N1YOfblBqRo4aBVbX9KEdVM3bw+xY+BOKCgDAJWXm5Gvo9I06eCJLdWtU0ewRHVWLofHtDkUFAOBycvKtemRWrLYfSVftal6aNaKjQvyrmB0LxaCoAABcSoHVptFzNmvtwZOq7u2hGcM6qmFAdbNj4RIoKgAAl2EYhsZ9t12//HH/nqkPtlfLMH+zY+EyKCoAAJcx6cc9+jr2sNws0of3tlXnhty/x95RVAAALmHKigP6eOVBSdKkAa3Uq0WwyYlQEhQVAIDTm7MhSZN+3CNJevHWprq7fbjJiVBSFBUAgFP7futRvThvuyRpZPeGeuS6hiYnQmlQVAAATmvJrjSNmbtFhiEN7lRPY2+JMjsSSomiAgBwSmv2n9DjX8SpwGbozrZ19VrfaG4y6IAoKgAApxOXdFoPzdqkvAKbbmoepLfvaiU3bjLokCgqAACnsjslQ0On/e9OyB/e21Ye7uzuHBVbDgDgNA4eP6sHPl2vjJwCxUTU1CcPxsjHkzshOzKKCgDAKRw5c073/3e9TpzNU/MQP00b2kFVvbgTsqOjqAAAHN6xzBwNnrpOR9Nz1CCgmmaN6Cj/Kp5mx0I5oKgAABzamew8PfjpBiWczFbdGlX0+UOdVKe6t9mxUE4oKgAAh3U2t0BDp2/UntRMBfp664uHOynEv4rZsVCOKCoAAIeUk2/VwzM3aUvyGdWo6qnPHuqkiNrVzI6FckZRAQA4nLwCmx7/PE5rD55UdW8PzRzWUU2CfM2OhQpAUQEAOJQCq02j52zWb3uOydvDTZ8Oaa/W4TXMjoUKQlEBADgMq83QmK+26qedqfJyd9PUB9urU4PaZsdCBaKoAAAcgs1maOy32/T91qPycLPoP4Pb6bomAWbHQgWjqAAA7J5hGHp5wQ59E3tY7m4WfXhvW/VsHmR2LFQCigoAwK4ZhqHXFu3W5+uTZLFI797dWr1bhpgdC5XEborKpEmTZLFY9PTTT5sdBQBgJwzD0Fs/x2va6kOSpDf7t1LfNnVNToXKZBdFZePGjfr444/VqlUrs6MAAOzIB0v3a/LyA5Kk1/pF6+4O4SYnQmUzvaicPXtWgwcP1tSpU1WzZk2z4wAA7MSUFQf03pK9kqSX+jTTA9dEmJwIZjC9qIwaNUp9+vRRz549rzhvbm6uMjIyivwAAJzP9NWHNOnHPZKk53pF6aFrG5icCGYx9f7XX375peLi4rRx48YSzT9x4kRNmDChglMBAMz0xfokTVi4S5I0+sZGGnVDI5MTwUymHVFJTk7WU089pc8//1w+Pj4les24ceOUnp5e+JOcnFzBKQEAlenb2MP6+/ztkqRHr2ugZ25qYnIimM1iGIZhxi+eP3++7rzzTrm7uxdOs1qtslgscnNzU25ubpHnipORkSF/f3+lp6fLz8+voiMDACrQgi1H9MzcLbIZ0tAu9fXq7c1lsVjMjoUKUJr9t2lf/fTo0UPbt28vMm3YsGFq2rSpxo4de8WSAgBwHt9vPVpYUu7tGK5XbqOk4DzTioqvr6+io6OLTKtWrZpq16590XQAgPNatO2onv5ys2yGNKh9uF7v11JubpQUnGf6VT8AANf1w/YUPfXl+SMpA2PCNLE/JQVFmXrVz18tX77c7AgAgEry4/YUPTlns6w2QwPahWnSgFaUFFyEIyoAgEr3047UwpLSv21dvXVXK7lTUlAMigoAoFL9sjNVT3wRpwKbob5tQvX2wNaUFFwSRQUAUGmW7ErTqD9Kyu2tQ/VPSgqugKICAKgUv+1J02OfxyrfaqhPqxC9d3drebizG8Ll8Q4BAFS4ZfHHNHJ23PmS0jJE/xrUhpKCEuFdAgCoUCv2Htejs2OVZ7Wpd3Sw3r+HkoKS450CAKgwK/ce18OzNimvwKZeLYL0wb1t5UlJQSnwbgEAVIhle47poT9KSs9mQfrw3naUFJQa7xgAQLlbsivt/Nc9BTbd3DxI/xncTl4e7HJQenY1Mi0AwPH9/Mc4KflWQ72jg/m6B1eFogIAKDcXhsUvsBm6rVWI3hvUhpKCq0JRAQCUi0XbjuqpL7fI+seIs/8cyDgpuHoUFQDAVVuw5YiemXv+Lsj929ZlWHyUG6ouAOCqfBd3uLCkDIwJo6SgXHFEBQBQZl9vStbz326TYUj3dAjXG3e2lBslBeWIIyoAgDL5ckNSYUkZ3KkeJQUVgiMqAIBS+3x9ov4+b4ckaUjnCI2/o4UsFkoKyh9FBQBQKrPWJuiVBTslScO61tcrtzWnpKDCUFQAACX28YoDmvjjHknSw9dG6sVbm1FSUKEoKgCAKzIMQ/9auk/vL9knSRp1Q0M9e3MUJQUVjqICALgswzA06ac9+njFQUnSc72iNOqGRiangqugqAAALslmMzRh4U7NXJsoSXr5tuYa0S3S5FRwJRQVAECxrDZD477bpq82HZbFIv2jX7QGd4owOxZcDEUFAHCRfKtNf/tqq77felRuFumdga3Vv12Y2bHggigqAIAicgusevKLzfplV5o83Cz61z1t1adViNmx4KIoKgCAQjn5Vj06O1Yr9h6Xl4ebJg9upx7NgsyOBRdGUQEASJKycgv00MxNWnvwpKp4umvqg+3VrXEds2PBxVFUAABKP5evYdM3KC7pjKp7e2ja0A7qGFnL7FgARQUAXN3Js7kaMn2DdhzJkJ+Ph2aN6KQ24TXMjgVIoqgAgEtLST+n+/+7XgeOZ6l2NS/NHtFJzUP9zI4FFKKoAICLOnQiS/f/d72OnDmnEH8fffZQJzUMqG52LKAIigoAuKDdKRl64NMNOnE2V5F1qmn2iI4Kq1nV7FjARSgqAOBiYhNPa9j0DcrIKVCzED/NGt5RAb7eZscCikVRAQAXsmrfcT0yK1bn8q1qH1FTnw7tIP8qnmbHAi6JogIALuLH7Ska/eVm5VsNXdckQFPub6eqXuwGYN94hwKAC/hqU7Je+HabbIbUp2WI3hvURl4ebmbHAq6IogIATu6/qw7qH4t3S5IGtQ/XG/1byt3NYnIqoGQoKgDgpAzD0HtL9umDpfskSY9c10DjejeVxUJJgeOgqACAE7LZDP3fol2asSZBkvRcryg9fn1DSgocDkUFAJxMXoFNz3+zVfO3HJUkvda3hR7oXN/cUEAZUVQAwIlk5xVo5GdxWrn3uDzcLHpnYGv1a1vX7FhAmVFUAMBJnMrK07AZG7U1+YyqeLrrP/e30w1RgWbHAq5KqYvK7t279eWXX2rVqlVKTExUdna2AgIC1LZtW/Xq1UsDBgyQtzcjHAJAZTp8OlsPTtugg8ezVKOqp6YP7aC29WqaHQu4ahbDMIySzBgXF6fnn39ev//+u7p27aqOHTsqNDRUVapU0alTp7Rjxw6tWrVKGRkZev755/X0009XeGHJyMiQv7+/0tPT5efH3T4BuKa9aZl68NMNSs3IUai/j2aN6KhGgb5mxwIuqTT77xIfURkwYICee+45ffPNN6pRo8Yl51u7dq3+9a9/6Z///KdefPHFEocGAJTepoRTGj5jozJyCtQ4sLpmjeioEP8qZscCyk2Jj6jk5+fL07Pk94MoyfyTJ0/W5MmTlZCQIElq0aKFXnnlFfXu3btEv4MjKgBc2dLdaXr88zjlFtgUE1FTnw5prxpVvcyOBVxRafbfJR4/uaQlJTs7u8Tzh4WFadKkSYqNjdWmTZt04403qm/fvtq5c2dJYwGAS/p6U7IemR2r3AKbbmwaqM9GdKKkwCmV6UYPPXr00JEjRy6avmHDBrVp06bEy7n99tt16623qnHjxmrSpIlef/11Va9eXevWrStLLABweoZhaMqKA3rum22y2gwNaBemjx+IURUvd7OjARWiTEXFx8dHrVq10ty5cyVJNptN48ePV7du3XTrrbeWKYjVatWXX36prKwsde7cuUzLAABnZrMZen3xbk36cY8k6dHuDfTOwFbydOfmgnBeZRpHZfHixfroo480fPhwLViwQAkJCUpMTNSiRYt08803l2pZ27dvV+fOnZWTk6Pq1atr3rx5at68ebHz5ubmKjc3t/BxRkZGWeIDgMP562izf7+1mR6+roHJqYCKV+YB30aNGqXDhw/rzTfflIeHh5YvX64uXbqUejlRUVHasmWL0tPT9c0332jIkCFasWJFsWVl4sSJmjBhQlkjA4BDysjJ18jZsVpz4KQ83Cx6665W6t8uzOxYQKUo8VU/f3b69Gk99NBDWrp0qd5++22tWLFC8+fP11tvvaXHH3/8qgL17NlTDRs21Mcff3zRc8UdUQkPD+eqHwBOKyX9nIZN36g9qZmq5uWu/9wfo+5NAsyOBVyVChlH5c+io6MVGRmpzZs3KzIyUg8//LDmzp2rxx9/XIsXL9bixYvLFFw6f77Ln8vIn3l7ezPqLQCXEZ+aqaHTNyglPUcBvt6aPrSDouv6mx0LqFRlOgNr5MiRWrlypSIjIwunDRo0SFu3blVeXl6JlzNu3DitXLlSCQkJ2r59u8aNG6fly5dr8ODBZYkFAE5j7YGTumvKGqWk56hhQDV991gXSgpcUpm++ikvI0aM0NKlS5WSkiJ/f3+1atVKY8eO1U033VSi1zPgGwBn9P3Wo3r2q63Ks9rUPqKm/stAbnAyFfLVT1JSkurVq1fiEEeOHFHdupe/tfinn35a4uUBgLMzDENTVx3UGz+cv/y4d3Sw3hvURj6ejJEC11Xir346dOigRx99VBs3brzkPOnp6Zo6daqio6P17bfflktAAHAFVpuhCQt3FZaUoV3q69/3taOkwOWV+IjK7t279Y9//EM33XSTfHx8FBMTo9DQUPn4+Oj06dPatWuXdu7cqXbt2umtt94q88BvAOBqcvKtembuFv24I1XS+TFSHro2UhaLxeRkgPlKfI7Ktm3b1KJFC+Xl5emHH37QqlWrlJiYqHPnzqlOnTpq27atevXqpejo6IrOXIhzVAA4utNZeXp41iZtSjwtL3c3vXN3a93ROtTsWECFKs3+u8RFxd3dXampqQoICFCDBg20ceNG1a5du1wClxVFBYAjSzqZraEzNujg8Sz5+njokwfaq3NDc/+uApWhQu6eXKNGDR08eFCSlJCQIJvNdnUpAcCFxSae1p3/Wa2Dx7MU4u+jb0Z2oaQAxSjxOSoDBgxQ9+7dFRISIovFovbt28vdvfiTvC4UGgDAxRZvS9EzX21RXoFNLUL9NG1oBwX5+ZgdC7BLJS4qn3zyifr376/9+/dr9OjRevjhh+Xr61uR2QDAqRiGoSkrDurNn85f2dOzWaD+dU9bVfMu823XAKdXqk/HLbfcIkmKjY3VU089RVEBgBLKt9r08vwd+nJjsqTzlx+/fFtzubtxZQ9wOWWq8dOnTy/vHADgtDJy8vX4Z3H6ff8JuVmkl29rrmFdI6/8QgBlKyoAgJI5fDpbw6Zv1L5jZ1XVy10f3ttWPZoFmR0LcBgUFQCoIFuTz2jEzE06cTZXQX7e+nQIdz8GSouiAgAV4KcdqXp67mbl5NvUNNhX04d1UIh/FbNjAQ6HogIA5cgwDP131SG98eNuGYZ0fVSA/n1fO1Xnyh6gTPjkAEA5ybfa9MqCnZqzIUmSdP819TT+9hbycC/x2JoA/oKiAgDl4HRWnh77PFbrDp6SxXL+xoIjunFjQeBqUVQA4CrtP3ZWI2ZuVOLJbFXzctcHXNkDlBuKCgBchZV7j2vUF3HKzClQWM0q+nRIB0UFMxgmUF4oKgBQBoZhaNbaRP3fol2y2gy1j6ipKQ/EqE51b7OjAU6FogIApZRvtWn89zv1+frzJ83eFROm1++MlrdH8TdqBVB2FBUAKIUz2Xl6/PM4rTlwUhaL9MItTfXIdQ04aRaoIBQVACihA8fP6qGZm3ToRJaqebnr/Xva6qbmnDQLVCSKCgCUwO/7Tujxz2OVkVOgujWq6L9D2qtZiJ/ZsQCnR1EBgMswDEOz1yVqwsLzJ83GRNTUx5w0C1QaigoAXEJugVWvzN+puZuSJUn929bVG/1byseTk2aBykJRAYBiHMvI0cjPYhWXdEZuFmksJ80CpqCoAMBfbEk+o0dnb1JaRq78fDz04X3t1L1JgNmxAJdEUQGAP/k29rDGzduuvAKbGgVW19QH2yuyTjWzYwEui6ICAJIKrDa98cMeTVt9SJLUs1mQ3hvUWr4+niYnA1wbRQWAyzudlacn5sRp9f6TkqTRPRrr6R6N5ebG+SiA2SgqAFzantQMPTxrk5JPnVNVL3e9e3dr3RIdYnYsAH+gqABwWT/tSNGYr7YqO8+q8FpVNPXB9moazCBugD2hqABwOTabofeX7tMHS/dJkro2qq1/39tONat5mZwMwF9RVAC4lPTsfD09d7OWxR+XJI3oFqlxvZvKw93N5GQAikNRAeAydh5N12OfxSnpVLa8Pdz0xp0tNSAmzOxYAC6DogLAJXwXd1jjvtuu3AKbwmtV0ZT7Y9Qi1N/sWACugKICwKnlFdj0j8W7NGttoiTp+qgAvT+ojWpU5XwUwBFQVAA4rbSMHD3+eZxiE09LYnwUwBFRVAA4pfUHT2rUF5t14myufH089P6gNurRLMjsWABKiaICwKkYhqFpqxP0xg+7ZbUZahrsqyn3x6g+9+sBHBJFBYDTyM4r0Nhvt2vh1qOSpL5tQjWxf0tV9eJPHeCo+PQCcAoHj5/VY5/FKT4tUx5uFr3Up5mGdKkvi4XzUQBHRlEB4PAWbTuqsd9sU1aeVQG+3vrP4HbqUL+W2bEAlAOKCgCHlVtg1RuLd2vmH5ced4yspX/f21aBfj4mJwNQXigqABxS8qlsPfFFnLYeTpckPX59Q425qQlD4QNOhqICwOEs2ZWmMV9tUUZOgfyreOq9Qa11Y1MuPQacEUUFgMMosNr09i/x+njFQUlSm/Aa+vd9bRVWs6rJyQBUFFOPkU6cOFEdOnSQr6+vAgMD1a9fP8XHx5sZCYCdSk3P0X1T1xeWlKFd6uurRztTUgAnZ2pRWbFihUaNGqV169bp119/VX5+vm6++WZlZWWZGQuAnfl93wn1+WCVNiScUnVvD/1ncDuNv6OFvDw4HwVwdhbDMAyzQ1xw/PhxBQYGasWKFbruuuuuOH9GRob8/f2Vnp4uPz+/SkgIoDJZbYY+/G2f/rV0nwxDahbip/8MbqdIRpkFHFpp9t92dY5Kevr5s/dr1Sp+/IPc3Fzl5uYWPs7IyKiUXAAq37HMHI2Zu1W/7z8hSbqnQ7jG39FCPp7uJicDUJnspqjYbDY9/fTT6tq1q6Kjo4udZ+LEiZowYUIlJwNQ2VbuPa4xX23RibN58vF00+v9WmpATJjZsQCYwG6++nnsscf0448/6vfff1dYWPF/kIo7ohIeHs5XP4CTyLfa9M9f9mrKigOSpKbBvvr3fW3VKNDX5GQAypPDffXzxBNPaNGiRVq5cuUlS4okeXt7y9vbuxKTAagsyaeyNfrLzdqcdEaSdP819fRSn+Z81QO4OFOLimEYevLJJzVv3jwtX75ckZGRZsYBYJIft6fo+W+3KTOnQL4+HnprQCv1bhlidiwAdsDUojJq1Ch98cUXWrBggXx9fZWamipJ8vf3V5UqVcyMBqAS5ORb9dqiXfp8fZIkqW29GvrgnrYKr8XYKADOM/UclUvdfn369OkaOnToFV/P5cmA49p/LFNPfLFZe1IzJUmP/XGvHk/u1QM4PYc5R8VOzuMFUIkMw9DXmw7r1e936ly+VXWqe+ndu9vouiYBZkcDYIfs4mRaAK4hIydfL83boe+3HpUkXdu4jv55d2sF+vqYnAyAvaKoAKgUGw6d0jNzt+jImXNyd7Pobzc30cjrGsrNrfivgAFAoqgAqGD5Vps+WLpPHy3bL5sh1atVVe/f00bt6tU0OxoAB0BRAVBhDp3I0tNzt2hr8hlJ0l0xYRp/RwtV9+ZPD4CS4a8FgHJ34YTZ8Qt3KjvPKj8fD73Rv6VuaxVqdjQADoaiAqBcnc7K04vztuvHHefHRbqmQS29e3cbhdZgbCQApUdRAVBuVu8/oTFfbVFaRq483Cx6tleUHr62gdw5YRZAGVFUAFy13AKr/vnLXn2y8qAkqUGdavrXPW3VMszf5GQAHB1FBcBV2ZuWqae/3KJdKRmSpPs61dNLfZqpqhd/XgBcPf6SACgTq83QtN8P6e1f4pVXYFPNqp56c0Ar3dwi2OxoAJwIRQVAqSWdzNazX2/VhoRTkqQbogL05oBWCvRjhFkA5YuiAqDEDMPQnA3J+sfiXcrOs6qal7teuq257ukQfsmbjALA1aCoACiRYxk5ev7bbVoef1yS1LF+Lb0zsLXq1a5qcjIAzoyiAuCKFm49qpcX7NCZ7Hx5ebjpuZujNLxbJJcdA6hwFBUAl3Q6K08vL9ihRdtSJEnRdf307t1t1CTI1+RkAFwFRQVAsZbFH9PYb7bpWGau3N0sGnVDIz15YyN5uruZHQ2AC6GoACgiIydfbyzerS83JkuSGgRU03t3t1Hr8BrmBgPgkigqAAotiz+mF7/brpT0HEnSsK71NfaWpvLxdDc5GQBXRVEBoPTsfP3fol36Nu6wJCmidlW9OaCVrmlQ2+RkAFwdRQVwcUt2penFedt1LDNXFos0rEuknusVpSpeHEUBYD6KCuCiTmflacLCnZq/5aik8zcSfOuuVmpfv5bJyQDgfygqgAv6aUeKXpq/UyfO5srNIj18bQM9c1MTzkUBYHcoKoALOXk2V698v1OL/xgXpXFgdb11Vyu1rVfT5GQAUDyKCuACDMPQom0pevX7nTqVlSd3N4tGdm+g0T0ay9uDoygA7BdFBXByR86c0yvzd2jpnmOSpKbBvnr7rtZqGeZvcjIAuDKKCuCkrDZDs9Ym6J2f45WVZ5Wnu0WPX99Io25oJC8PRpcF4BgoKoAT2pOaoRe+3a4tyWckSTERNTWpf0s15h49ABwMRQVwIjn5Vn2wdJ8+WXlQBTZDvt4eer53Uw3uWE9u3OkYgAOiqABOYs2BE3rxu+1KOJktSerVIkgT7ohWsL+PyckAoOwoKoCDO5Odp9cX79bXseeHvw/y89aEO6J1S3SwyckA4OpRVAAHZRiGFm5L0f8t3KkTZ/MkSfdfU0/P39JUfj6eJqcDgPJBUQEc0KETWXplwQ6t2ndCktQosLom9W/J8PcAnA5FBXAgOflWTV5+QJNXHFBegU1e7m56/IaGeuz6hgzcBsApUVQAB7E8/phe/X6nEv84WfbaxnX0f32jFVmnmsnJAKDiUFQAO5eSfk7/t3CXftyRKun8ybKv3NZCt7YMlsXCJccAnBtFBbBT+VabZqxO0HtL9io7zyp3N4uGdamvp29qourefHQBuAb+2gF2aGPCKb00b4fi0zIlnR9Z9rW+0Woe6mdyMgCoXBQVwI6cPJurST/uKRwTpWZVT43r3Ux3xYQxsiwAl0RRAexAvtWmz9Yl6t1f9yozp0CSdG/HcD3fq6lqVvMyOR0AmIeiAphs9f4TmrBwp/amnZUkNQ/x02v9ohUTUdPkZABgPooKYJLkU9l644fdhVfz1KzqqWd7RemeDvXkztc8ACCJogJUunN5Vk1ZcUBTVhxQboFNbhbpgWsi9MxNTVSjKl/zAMCfUVSASmIYhn7ckarXF+/WkTPnJEnXNKil8Xe0UNNgruYBgOJQVIBKsCc1QxO+36W1B09KkurWqKK/92mm3tEM2gYAl0NRASrQqaw8/WvJXn22PklWmyFvDzeN7N5QI7s3VBUv7s0DAFfiZuYvX7lypW6//XaFhobKYrFo/vz5ZsYByk1ugVWfrDyg7m8v08y1ibLaDPWODtaSMd31zE1NKCkAUEKmHlHJyspS69atNXz4cPXv39/MKEC5uHAeysQfdyv51PnzUJqF+OmlPs3UtVEdk9MBgOMxtaj07t1bvXv3NjMCUG42J53W64t3a1PiaUlSoK+3nu0VpQHtwrjcGADKyKHOUcnNzVVubm7h44yMDBPTAOcdPp2tt36K1/dbj0qSfDzd9Mh1DfXodQ1UjZsHAsBVcai/ohMnTtSECRPMjgFIkjJz8vWf5Qf06e+HlFdgk8Ui9W8bpud6RSnY38fseADgFByqqIwbN05jxowpfJyRkaHw8HATE8EVFVhtmrspWe/+slcns/IknR8P5aU+zRVd19/kdADgXByqqHh7e8vb29vsGHBRhmHopx2pevuXeB08niVJiqxTTS/e2kw9mwUyHgoAVACHKiqAWdYcOKE3f4rX1uQzks7fl2d0j8Ya3ClCXh6mXuUPAE7N1KJy9uxZ7d+/v/DxoUOHtGXLFtWqVUv16tUzMRlw3s6j6Xrrp3it2HtcklTVy10PdYvUw9c1kK+Pp8npAMD5mVpUNm3apBtuuKHw8YXzT4YMGaIZM2aYlAqQkk5m65+/xmvBlvNX8ni4WXRfp3p68sbGCvDl60cAqCymFpXrr79ehmGYGQEo4sTZXP37t/36fH2i8q3n35u3tw7V325qovp1qpmcDgBcD+eoAJLO5hbov6sOaurKg8rKs0qSrm1cR2NvacqVPABgIooKXFp2XoFmrU3UxysO6HR2viSpVZi/Xrilqbow5D0AmI6iApeUk2/V5+uTNHn5fp04e34slAZ1qulvN0fp1pbBXGoMAHaCogKXkltg1dyNyfpo2X6lZZy/HUO9WlX1VI/G6tsmVB7uXGoMAPaEogKXkG+16etNh/Xv3/bpaHqOJKlujSp68sZGGhATJk8KCgDYJYoKnFqB1aZ5m4/og9/2KfnUOUlSkJ+3nrihke7uEC5vD3eTEwIALoeiAqdUYLVp0bYUfbB0nw6eOD/cfZ3qXnrs+kYa3KmefDwpKADgCCgqcCr5VpvmxR3Rf5bvV8LJbEnnh7sf2b2hHugcoapevOUBwJHwVxtOISffqq9jD2vK8gM6cub8Vzw1q3pqRLdIDe0aqerevNUBwBHx1xsO7VyeVV9sSNInKw8UXsVTp7q3HrkuUoM7RagaBQUAHBp/xeGQzuYWaPbaRP131UGdzDo/DkqIv49Gdm+oQR3COQcFAJwERQUOJT07XzPWJGja6kNKP3d+JNnwWlX0+PWN1L9dXa7iAQAnQ1GBQ0hNz9H01Yf0+foknc0tkCQ1CKimUdc30h1tQhkHBQCcFEUFdm1fWqY+WXlQ87ccKbybcVSQr564sZFubRkidzeGugcAZ0ZRgd0xDEObEk/r4xUHtGT3scLpHSNraWT3Brq+SaDcKCgA4BIoKrAbNpuhX3en6eMVBxSXdEaSZLFIvZoH65HuDdSuXk1zAwIAKh1FBabLLbBqXtwRfbLqoA4ePz+KrJe7mwbE1NVD1zZQw4DqJicEAJiFogLTnMrK05wNSZqxJkHHM8+PgeLn46H7r4nQ0K71FejrY3JCAIDZKCqodPGpmZq++pDmbT6i3AKbpPNjoIzoFql7OtZjFFkAQCH2CKgUNpuhZfHHNG31Ia3ef7Jwesu6/hrWtb5uaxUqLw8uMQYAFEVRQYU6m1ugbzYla8aahMKbBLpZpFuigzW8a6RiImrKYuEKHgBA8SgqqBDJp7I1c02C5m5MVuYfA7T5+Xjo3o719EDnCIXVrGpyQgCAI6CooNzYbIZ+339Cn61L1JLdabKdH59NDQKqaViX+urfLoybBAIASoW9Bq7a6aw8fRN7WJ+vTyz8ekeSrm1cR8O7Rap74wAGaAMAlAlFBWViGIY2J5/RZ+sStWhbivL+uHrH19tD/dvV1f3XRKhxkK/JKQEAjo6iglLJzivQgi1H9dm6RO08mlE4vUWon+6/JkJ3tA7l6x0AQLlhj4IS2ZeWqc/XJ+nb2MOFJ8d6ebjptlYheuCaCLUJr8HVOwCAckdRwSVl5RZo8bYUzd2UrNjE04XTI2pX1f2dInRXTJhqVvMyMSEAwNlRVFCEYRiKSzqjrzYma9G2o8rKs0qS3N0surFpoB64JkLdGtXh5FgAQKWgqECSdOJsrubFHdHcTcnaf+xs4fTIOtU0sH2Y7moXpkA/7r0DAKhcFBUXZrUZWrn3uOZuTNaS3Wkq+GPgEx9PN93aMkSD2oerY2Qtzj0BAJiGouKC9qZlat7mI5oXd0SpGTmF01uH19Cg9uG6vXWIfH08TUwIAMB5FBUXcSwjR99vParv4o5oV8r/LiuuWdVTd7YN06AO4YoKZtwTAIB9oag4sazcAv2yK1XfxR3R6v0nCoe093S36PqoQPVvW1c3NguUt4e7uUEBALgEioqTKbDatPrASc3ffEQ/7UjVuXxr4XMxETXVr21d3dYyhMuKAQAOgaLiBGy288PZL96WooXbjup4Zm7hc/VrV9WdbcPUr22oImpXMzElAAClR1FxUIZhaOvhdC3aelQ/bE/R0fT/nRRbs6qnbm8dqjvb1mXEWACAQ6OoOBDDMLT9SLoWb0vRom0pOnLmXOFz1b09dFPzIPVpGaLuUQHydHczMSkAAOWDomLnDMPQzqMZWrw9RYu3pSjpVHbhc1W93NWzWZD6tApR9yYB8vHkpFgAgHOhqNghq83Q5qTT+mVXmn7ZmaqEk/8rJ1U83dWjWaBuaxWi66MCKScAAKdGUbETOflWrd5/Qr/sTNPSPWk6cTav8DkfTzfd2DRQfVqG6oamAarqxWYDALgG9ngmOpOdp9/2HNMvO9O0ct9xZef971JiXx8P9WgaqJtbBKt7kwBV82ZTAQBcD3u/SpZ0MltL96Tpl51p2pBwStYLo7BJCvH30c3Ng3Rzi2B1jKzFCbEAAJdHUalgOflWbTh0Ssvjj2t5/DEdPJFV5Pmmwb6F5aRFqB+XEgMA8CcUlQqQfCpby/ce1/I9x7TmwMkio8O6u1nUPqKmbmoepJubB6te7aomJgUAwL5RVMpBboFVGw+d1vL4Y1oWf0wHjhc9ahLo660bogJ1fVSAujauIz/uTAwAQInYRVH56KOP9Pbbbys1NVWtW7fWhx9+qI4dO5od65KsNkO7jmZo9YETWr3/hDYmnFJOvq3weXc3i2Lq1dT1TQN0fZNANQvx5SsdAADKwPSiMnfuXI0ZM0ZTpkxRp06d9P7776tXr16Kj49XYGCg2fEknR907eCJLK3Zf0Kr95/U2oMnlX4uv8g8Ab7eur5JgK6PClS3xnXkX4WjJgAAXC2LYRjGlWerOJ06dVKHDh3073//W5Jks9kUHh6uJ598Ui+88MJlX5uRkSF/f3+lp6fLz8+vXHOlpudo9f4TWn3ghNbsP6nUjJwiz1f39tA1DWqpS8M66tqojpoEVeeoCQAAJVCa/bepR1Ty8vIUGxurcePGFU5zc3NTz549tXbt2ovmz83NVW7u/+4MnJGRUSG5pq8+pAkLdxWZ5uXuppiImuraqLa6NKqjVnX95cHlwwAAVChTi8qJEydktVoVFBRUZHpQUJD27Nlz0fwTJ07UhAkTKjxXdF1/uVmklnX91aVRHXVtWEft69dkuHoAACqZ6eeolMa4ceM0ZsyYwscZGRkKDw8v99/TNryGNr9yM+eZAABgMlOLSp06deTu7q60tLQi09PS0hQcHHzR/N7e3vL29q7wXB7ubvKvwtc6AACYzdS9sZeXl2JiYrR06dLCaTabTUuXLlXnzp1NTAYAAOyB6V/9jBkzRkOGDFH79u3VsWNHvf/++8rKytKwYcPMjgYAAExmelEZNGiQjh8/rldeeUWpqalq06aNfvrpp4tOsAUAAK7H9HFUrkZFjqMCAAAqRmn235wxCgAA7BZFBQAA2C2KCgAAsFsUFQAAYLcoKgAAwG5RVAAAgN2iqAAAALtFUQEAAHaLogIAAOyW6UPoX40Lg+pmZGSYnAQAAJTUhf12SQbHd+iikpmZKUkKDw83OQkAACitzMxM+fv7X3Yeh77Xj81m09GjR+Xr6yuLxVKuy87IyFB4eLiSk5Od8j5Czr5+EuvoDJx9/STW0Rk4+/pJ5b+OhmEoMzNToaGhcnO7/FkoDn1Exc3NTWFhYRX6O/z8/Jz2jSc5//pJrKMzcPb1k1hHZ+Ds6yeV7zpe6UjKBZxMCwAA7BZFBQAA2C2KyiV4e3vr1Vdflbe3t9lRKoSzr5/EOjoDZ18/iXV0Bs6+fpK56+jQJ9MCAADnxhEVAABgtygqAADAblFUAACA3aKoAAAAu+XSReWjjz5S/fr15ePjo06dOmnDhg2Xnf/rr79W06ZN5ePjo5YtW+qHH36opKSlM3HiRHXo0EG+vr4KDAxUv379FB8ff9nXzJgxQxaLpciPj49PJSUuvfHjx1+Ut2nTppd9jaNsvwvq169/0TpaLBaNGjWq2PntfRuuXLlSt99+u0JDQ2WxWDR//vwizxuGoVdeeUUhISGqUqWKevbsqX379l1xuaX9HFeky61jfn6+xo4dq5YtW6patWoKDQ3Vgw8+qKNHj152mWV5r1ekK23HoUOHXpT3lltuueJy7WU7Xmn9ivtMWiwWvf3225dcpr1tw5LsI3JycjRq1CjVrl1b1atX14ABA5SWlnbZ5Zb1M3wlLltU5s6dqzFjxujVV19VXFycWrdurV69eunYsWPFzr9mzRrde++9GjFihDZv3qx+/fqpX79+2rFjRyUnv7IVK1Zo1KhRWrdunX799Vfl5+fr5ptvVlZW1mVf5+fnp5SUlMKfxMTESkpcNi1atCiS9/fff7/kvI60/S7YuHFjkfX79ddfJUkDBw685GvseRtmZWWpdevW+uijj4p9/q233tIHH3ygKVOmaP369apWrZp69eqlnJycSy6ztJ/jina5dczOzlZcXJxefvllxcXF6bvvvlN8fLzuuOOOKy63NO/1inal7ShJt9xyS5G8c+bMuewy7Wk7Xmn9/rxeKSkpmjZtmiwWiwYMGHDZ5drTNizJPuKZZ57RwoUL9fXXX2vFihU6evSo+vfvf9nlluUzXCKGi+rYsaMxatSowsdWq9UIDQ01Jk6cWOz8d999t9GnT58i0zp16mQ8+uijFZqzPBw7dsyQZKxYseKS80yfPt3w9/evvFBX6dVXXzVat25d4vkdeftd8NRTTxkNGzY0bDZbsc870jaUZMybN6/wsc1mM4KDg4233367cNqZM2cMb29vY86cOZdcTmk/x5Xpr+tYnA0bNhiSjMTExEvOU9r3emUqbh2HDBli9O3bt1TLsdftWJJt2LdvX+PGG2+87Dz2vA0N4+J9xJkzZwxPT0/j66+/Lpxn9+7dhiRj7dq1xS6jrJ/hknDJIyp5eXmKjY1Vz549C6e5ubmpZ8+eWrt2bbGvWbt2bZH5JalXr16XnN+epKenS5Jq1ap12fnOnj2riIgIhYeHq2/fvtq5c2dlxCuzffv2KTQ0VA0aNNDgwYOVlJR0yXkdeftJ59+zn332mYYPH37ZG3A62ja84NChQ0pNTS2yjfz9/dWpU6dLbqOyfI7tTXp6uiwWi2rUqHHZ+UrzXrcHy5cvV2BgoKKiovTYY4/p5MmTl5zXkbdjWlqaFi9erBEjRlxxXnvehn/dR8TGxio/P7/INmnatKnq1at3yW1Sls9wSblkUTlx4oSsVquCgoKKTA8KClJqamqxr0lNTS3V/PbCZrPp6aefVteuXRUdHX3J+aKiojRt2jQtWLBAn332mWw2m7p06aLDhw9XYtqS69Spk2bMmKGffvpJkydP1qFDh3TttdcqMzOz2PkddftdMH/+fJ05c0ZDhw695DyOtg3/7MJ2KM02Ksvn2J7k5ORo7Nixuvfeey97k7fSvtfNdsstt2jWrFlaunSp3nzzTa1YsUK9e/eW1Wotdn5H3o4zZ86Ur6/vFb8SsedtWNw+IjU1VV5eXhcV6CvtIy/MU9LXlJRD3z0ZVzZq1Cjt2LHjit+Hdu7cWZ07dy583KVLFzVr1kwff/yxXnvttYqOWWq9e/cu/O9WrVqpU6dOioiI0FdffVWi/7txNJ9++ql69+6t0NDQS87jaNvQleXn5+vuu++WYRiaPHnyZed1tPf6PffcU/jfLVu2VKtWrdSwYUMtX75cPXr0MDFZ+Zs2bZoGDx58xZPW7XkblnQfYSaXPKJSp04dubu7X3QGc1pamoKDg4t9TXBwcKnmtwdPPPGEFi1apGXLliksLKxUr/X09FTbtm21f//+CkpXvmrUqKEmTZpcMq8jbr8LEhMTtWTJEj300EOlep0jbcML26E026gsn2N7cKGkJCYm6tdff73s0ZTiXOm9bm8aNGigOnXqXDKvo27HVatWKT4+vtSfS8l+tuGl9hHBwcHKy8vTmTNnisx/pX3khXlK+pqScsmi4uXlpZiYGC1durRwms1m09KlS4v8H+mfde7cucj8kvTrr79ecn4zGYahJ554QvPmzdNvv/2myMjIUi/DarVq+/btCgkJqYCE5e/s2bM6cODAJfM60vb7q+nTpyswMFB9+vQp1escaRtGRkYqODi4yDbKyMjQ+vXrL7mNyvI5NtuFkrJv3z4tWbJEtWvXLvUyrvRetzeHDx/WyZMnL5nXEbejdP4oZ0xMjFq3bl3q15q9Da+0j4iJiZGnp2eRbRIfH6+kpKRLbpOyfIZLE9glffnll4a3t7cxY8YMY9euXcYjjzxi1KhRw0hNTTUMwzAeeOAB44UXXiicf/Xq1YaHh4fxzjvvGLt37zZeffVVw9PT09i+fbtZq3BJjz32mOHv728sX77cSElJKfzJzs4unOev6zdhwgTj559/Ng4cOGDExsYa99xzj+Hj42Ps3LnTjFW4or/97W/G8uXLjUOHDhmrV682evbsadSpU8c4duyYYRiOvf3+zGq1GvXq1TPGjh170XOOtg0zMzONzZs3G5s3bzYkGe+++66xefPmwiteJk2aZNSoUcNYsGCBsW3bNqNv375GZGSkce7cucJl3HjjjcaHH35Y+PhKn+PKdrl1zMvLM+644w4jLCzM2LJlS5HPZm5ubuEy/rqOV3qvV7bLrWNmZqbx7LPPGmvXrjUOHTpkLFmyxGjXrp3RuHFjIycnp3AZ9rwdr/Q+NQzDSE9PN6pWrWpMnjy52GXY+zYsyT5i5MiRRr169YzffvvN2LRpk9G5c2ejc+fORZYTFRVlfPfdd4WPS/IZLguXLSqGYRgffvihUa9ePcPLy8vo2LGjsW7dusLnunfvbgwZMqTI/F999ZXRpEkTw8vLy2jRooWxePHiSk5cMpKK/Zk+fXrhPH9dv6effrrw3yIoKMi49dZbjbi4uMoPX0KDBg0yQkJCDC8vL6Nu3brGoEGDjP379xc+78jb789+/vlnQ5IRHx9/0XOOtg2XLVtW7PvywjrYbDbj5ZdfNoKCggxvb2+jR48eF613RESE8eqrrxaZdrnPcWW73DoeOnTokp/NZcuWFS7jr+t4pfd6ZbvcOmZnZxs333yzERAQYHh6ehoRERHGww8/fFHhsOfteKX3qWEYxscff2xUqVLFOHPmTLHLsPdtWJJ9xLlz54zHH3/cqFmzplG1alXjzjvvNFJSUi5azp9fU5LPcFlY/vhlAAAAdsclz1EBAACOgaICAADsFkUFAADYLYoKAACwWxQVAABgtygqAADAblFUAACA3aKoAAAAu0VRAQAAdouiAgAA7BZFBYDdOH78uIKDg/XGG28UTluzZo28vLwuuvs1ANfAvX4A2JUffvhB/fr105o1axQVFaU2bdqob9++evfdd82OBsAEFBUAdmfUqFFasmSJ2rdvr+3bt2vjxo3y9vY2OxYAE1BUANidc+fOKTo6WsnJyYqNjVXLli3NjgTAJJyjAsDuHDhwQEePHpXNZlNCQoLZcQCYiCMqAOxKXl6eOnbsqDZt2igqKkrvv/++tm/frsDAQLOjATABRQWAXXnuuef0zTffaOvWrapevbq6d+8uf39/LVq0yOxoAEzAVz8A7Mby5cv1/vvva/bs2fLz85Obm5tmz56tVatWafLkyWbHA2ACjqgAAAC7xREVAABgtygqAADAblFUAACA3aKoAAAAu0VRAQAAdouiAgAA7BZFBQAA2C2KCgAAsFsUFQAAYLcoKgAAwG5RVAAAgN2iqAAAALv1/0CbRkfB6HfgAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "def numerical_diff(f, x):\n",
    "    h = 1e-4\n",
    "    return (f(x+h) - f(x-h)) / (2*h)\n",
    "\n",
    "def function_1(x):\n",
    "    return 0.01*x**2 + 0.1*x\n",
    "\n",
    "x = np.arange(0, 20, 0.1)\n",
    "y = function_1(x)\n",
    "\n",
    "\n",
    "print(numerical_diff(function_1, 5))\n",
    "print(numerical_diff(function_1, 10))\n",
    "\n",
    "plt.title(\"function_1\")\n",
    "plt.xlabel(\"x\")\n",
    "plt.ylabel(\"f(x)\")\n",
    "plt.plot(x, y)\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 偏导数\n",
    "- 导数是函数在某一点的变化率\n",
    "- 偏导数是函数在某一点的某一方向的变化率\n",
    "$$\n",
    "f(x_0,x_1) = x_0^2 + x_1^2\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "6.00000000000378\n",
      "7.999999999999119\n"
     ]
    }
   ],
   "source": [
    "def function_2(x):\n",
    "    return x[0]**2+x[1]**2\n",
    "\n",
    "def function_temp1(x0):\n",
    "    return x0*x0+4.0**2.0\n",
    "\n",
    "def function_temp2(x1):\n",
    "    return 3.0**2.0+x1*x1\n",
    "    \n",
    "print(numerical_diff(function_temp1,3.0))\n",
    "print(numerical_diff(function_temp2,4.0))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.4 梯度\n",
    "梯度表示的是在当前点变化最大的方向，比如一个函数增长或者下降最大的方向，是所有偏导数的集合"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[6. 8.]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "def function_2(x):\n",
    "    return x[0]**2+x[1]**2\n",
    "\n",
    "def numerical_gradient(f,x):\n",
    "    h = 1e-4 # 0.0001\n",
    "    grad = np.zeros_like(x)\n",
    "    \n",
    "    for idx in range(x.size):\n",
    "        tmp_val = x[idx]\n",
    "        # f(x+h)的计算\n",
    "        x[idx] = tmp_val + h\n",
    "        fxh1 = f(x)\n",
    "\n",
    "        # f(x-h)的计算\n",
    "        x[idx] = tmp_val - h\n",
    "        fxh2 = f(x)\n",
    "\n",
    "        # 梯度计算\n",
    "        grad[idx] = (fxh1 - fxh2)/(2*h)\n",
    "        x[idx] = tmp_val # 还原值\n",
    "\n",
    "    return grad\n",
    "\n",
    "# numerical_gradient(function_2,np.array([3.0,4.0]))\n",
    "print(numerical_gradient(function_2,np.array([3.0,4.0])))\n",
    "\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
